Theory of Bayesian Optimization

Consider rolling a die with k sides, labeled a1, a2, ..., and ak, respectively. Let P(aj) be the ‘probability’ that a particular side aj appears after rolling the die. Before attempting to calculate P(aj), it is necessary to clarify the meaning of the word ‘probability’. In other words, we must specify what the number P(aj) quantifies. In statistics, the concept of ‘probability’ is formally interpreted in one of two ways. In the frequentist interpretation of probability, P(aj) is the fraction of times that aj appears in a very large number of die rolls. In the Bayesian interpretation of probability, P(aj) is the extent to which we believe that the number aj will appear prior to rolling the die. For the case of a die with k sides, there is little difference between the frequentist and Bayesian interpretation of probability. Given that the die is not biased in any way, we would set P(aj) = 1/k in both the frequentist and Bayesian interpretations. However, a major difference between the frequentist and Bayesian interpretation of probability arises when we consider so-called learning-type problems, in which new information on the system becomes available over time. For example, consider a robot whose job is to sort oranges from lemons. Suppose that the robot is presented with a fruit, and that the robot has no useful information to help distinguish between oranges and lemons. For example, the robot does not know that round fruit are more likely to be oranges rather than lemons. In this case, the robot would set P (o) = 1/2 and P(l) = 1/2, where P(o) and P(l) are the probabilities that the fruit is an orange or lemon, respectively. Now, suppose that new information is loaded into the robot’s memory from an external source, namely